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FOREWORD 


The  study  described  in  this  report  extends  mathematical 
methods  to  cover  cases  that  are  of  specific  interest  at  the  U.  S. 
Naval  Ordnance  Test  Station.  The  derivation  of  the  theory  is 
followed  by  its  application  to  the  analysis  of  data  that  must  be 
obtained  in  the  evaluation  of  weapon  systems. 

This  study  was  made  early  in  1961  under  departmental  over¬ 
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INTRODUCTION 

This  report  extends  certain  least-squares  methods  currently  in  use  at  the  Naval  Ordnance 
Test  Station  (NOTS)  to  cover  the  case  of  correlated  data.  This  extension  makes  possible,  for 
instance,  the  use  of  derived  azimuths  and  elevations  in  an  Askania  solution  for  space  position. 
The  general  theory  of  least  squares  is  given  in  the  literature  (Ref.  1—3).  This  report  contains  in¬ 
dependent  derivations  pertaining  to  certain  cases  of  special  interest  at  NOTS.  A  presentation  of 
the  theory  is  followed  by  detailed  discussions  of  the  applications  to  the  Askania  cinetheodolite 
solution  and  curve  fitting  of  space-position  data.  References  to  other  local  applications  are 
given,  but  the  specific  results  for  these  are  not  presented. 


DERIVATION  OF  THE  THEORY 


THE  COVARIANCE  MATRIX  OF  A  LINEAR  COMBINATION 
OF  RANDOM  VARIABLES 

Theorem  1.  Suppose 

U  =  DV  where 
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the  are  random  variables,  and  the  dij  are  constants.  Further  define 
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Here  ouv  =  cov (u,u). 

Then  a  =  DSDT . 

Proof.  If  A'i,  X2 
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are  constants  with 
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then  by  talcing  expected  values  we  obtain 

RT'V ’ £[(t‘A)(|‘A)]  ■  £(J,“W') ■  &****> 

E(  h)E(  T2 )  =  E(X  El  £  4,*,)  =  t  £  bjEiXj)  =  £  a^EiXJFAX,)  (2) 

\i=l  /  \;=1  /  i=l  ;-l  i,/=l 

In  obtaining  Eq.  1  and  2,  the  linearity  of  the  expected-value  operator  has  been  used  and  the  prod¬ 
ucts  of  sums  have  been  written  as  double  sums.  Noting  that  by  definition  auv  =  E(uv)  “  E(u)E(v), 
the  subtraction  of  Eq.  2  from  Eq.  1  gives 

n 

°TXT1  =  2  afij°XiXj  O) 

t./=l 


and  putting  =  rf;;  and  =  dpi  so  that 


Ti  =  ui  =  duXi  +  di2X 2  +  .  •  •  +  dinXn 

T2  =  up  =  dpXXl  +  dp 2X2  +  .  .  .  +  dpnXn 


gives,  by  virtue  of  Eq.  3, 


°H“p  =  Z  dlidpiaXiXj 
*,/=  1 

From  direct  calculation  of  the  fp  element  of  DSDT,  denoted  by  ( DSDT)ip ,  one  obtains 

( DSDT)lp=  £  diidpjcrXiX 


A  comparison  of  Eq.  4  and  5  gives 

Z?SZ)r  =  (<fyUp) 

as  required. 


(4) 

(5) 


A  FORMULA  FOR  THE  DIFFERENTIATION  OF  A 
QUADRATIC  FORM 


The  result  obtained  here  is  well  known,  but  is  established  in  a  convenient  form  for  use  in 
deriving  the  normal  equations  in  the  following  section. 

Theorem  2.  Let 
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F(U)  =(LU  +  OtP(LU  +  C) 
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—  =  2  LTP(LU  +  C) 
dV 

Proof.  It  can  be  verified  by  expansion  and  differentiation  that 

d  . r. 


and  W  is  a  column  vector;  also 


if  K  is  a  row  vector. 

By  the  Distributive  Law 
rt  n\  t  rT  .  riT  r  T- 


—  (WtBW)  =  2BW 
dW ■ 


-KW  =  K1 


F{U)  =  (CT  +  UT  L  r)P(LU  +  C)  =  ClPC  +  C1  PLU  +  Ul  L‘  PLU  +  U‘L‘PC 
Since  CT PLU  is  a  1  x  1  matrix  and  as  such  is  symmetric,  it  follows  that 


CtPLV  =  U  Li P1 C  -  U  L  PC 


Thus, 


F(U)  =  CTPC  +  2CT  PLU  +  Ut(LtPL)U 

Hence,  by  the  formulas  already  derived. 


—  =  2  LTPLU  +  2  (CtPL)t  =  2  LtP(LU  +  0 
dU 


as  required. 


THE  GENERALIZED  LEAST-SQUARES  CRITERION:  DERIVATION 
AND  DISCUSSION  OF  NORMAL  EQUATIONS 

Let  it  be  required  to  determine  the  parameters  ui,  uy,  .  .  .  u*  from  measurements  m  j,  m2, 
.  .  .  m.  with  covariance  matrix  M.  Define 


i  m  j  —  ai  fi  ] 

my  —  ay  Vy 


Uy 

u  =  . 
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Here  the  aiy  qy  and  are  constants,  U  is  positive  definite,  and  Q  is  of  rank  k.  The  latter 
assumption  implies  that  there  are  sufficient  independent  data  to  solve  the  problem. 

It  can  be  shown  (Ref.  2  and  3)  that  the  maximum-likelihood  estimate  of  uj,  u2,  .  .  .  uk  under 
the  assumption  of  normally  distributed  errors  in  the  is  given  by 

G(U)  =  (V  -  QU)TM~HV  -  QU)  =  minimum  (6) 

This  criterion  is  taken  as  the  generalized  least-squares  criterion  for  correlated  data. 

By  Theorem  2, 

—  =  2(.Q')TM~1(V  -QU) 

dV 

Equating  to  zero  gives  the  normal  equations 

AUmQTM~lV  (7) 

where 

A  =  QrM~lQ 

We  show  that  A  is  non-singular.  Consider  the  quadratic  form  H(X)  =  XTM~l  X,  which  is  posi¬ 
tive  definite.  If  we  perform  a  linear  transformation,  X  =  QY ,  there  results 

H(QY)  =  YT(QTM-1Q)Y 

By  a  theorem  (Ref.  4)  from  linear  algebra, 

rank  Q  +  nullity  Q  =  number  of  columns  of  Q 

Since  rank  Q  =  number  of  columns  of  Q  =  k,  it  follows  that  nullity  Q  =  0.  Thus  QY  can  be  zero 
only  if  Y  =  0.  Hence  H(QY),  when  regarded  as  a  quadratic  form  in  Y,  is  positive  definite; 

Q  M  Q  is  a  positive-definite  matrix  and,  as  such,  has  only  positive  eigenvalues.  That  is  to  say, 
QTM  1 Q  is  non-singular.  (Note  that  if  Q  had  rank  less  than  k,  then  A  would  be  singular,  since  the 
rank  of  a  product  is  at  most  equal  to  the  rank  of  any  factor.) 

Equation  7  can  therefore  be  solved  uniquely  for  U  as  follows: 

U  =  A~lQTM~lV  (8) 

In  order  to  prove  that  Eq.  8  actually  furnishes  a  minimum  of  the  expression  for  G(U),  Eq.  6,  let  U o 
be  the  solution  given  by  Eq,  8.  That  is,  let  Uq  =  A~lQTM~lV  and  let  U*  be  any  other  real  column 
vector  having  dimension  k.  It  follows  by  direct  expansion,  making  use  of  this  expression  for  U o, 
that 

CUT*)  -  a  Go)  =  ((/*-  U0)TA(U*  -  G0) 

Since  it  has  already  been  shown  that  A  is  positive  definite,  it  is  concluded  that  V o  does  indeed  i 
furnish  a  minimum  of  Eq.  6. 

If  S(j  is  the  variance— covariance  matrix  of  U,  we  have  by  a  direct  application  of  Theorem  1 
Sy  =  {A~1QTM~1)M(A~l  QtM~1)t  =  A~1(QTM-1Q)A~1 
or 

Sv  *A~l  (9) 
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Thus  the  entire  least-squares  solution  consists  of  computing  Sy  by  Eq.  9  and  U  by  Eq.  8.  The 
application  of  this  solution  to  the  various  specific  situations  merely  consists  of  properly  defining 
the  matrix  Q  and  the  quantities  at.  This  is  done  in  detail  for  the  Aslcania  cinetheodolite  solution 
and  the  application  to  curve  fitting  of  space-position  data.  References  are  given  for  other  appli¬ 
cations. 


APPLICATIONS 

APPLICATION  TO  THE  ASKANIA  CINETHEODOLITE  SOLUTION 

For  a  more  detailed  explanation  of  the  notation  used  in  this  section  the  reader  may  consult 
Ref.  5.  Let  r  be  the  number  of  Askania  stations.  Then,  n  =  2r.  It  is  assumed  that  the  ith  Askania 
station  determines  azimuth  and  elevation  measurements  Ai  and  E ;  with  covariance  matrix 

(2  ) 

°*t  aWi 

°AiEi  oi.  ] 

and  that  the  measurement  errors  from  a  given  station  are  independent  of  those  from  any  other  sta¬ 
tion.  Of  course,  since  and  £,•  are  assumed  correlated  here,  it  would  be  permissible  to  use 
azimuths  and  elevations  derived  from  other  primary  measurements  rather  than  direct  Askania  read¬ 
ings.  Under  these  conditions, 


(i  =  1,  2,  .  .  .  r) 


We  have 

mi  =  ith  azimuth  reading,  i  -  1,  2,  .  .  .  r 

mi  =  (i  —  r)th  elevation  reading,  i  =  r  +  1,  .  .  .  2r 

In  order  to  obtain  iW-1  as  required  for  the  normal  equations,  one  may  follow  the  analysis1  of  Ref.  6 
with  the  following  result,  which  is  easily  verified  by  direct  calculation. 

1  The  use  of  this  procedure,  which  finds  Af  1  in  terms  of  submatrices  that  are  diagonal,  was  suggested 
by  Mrs.  D.  Saitz  of  the  Test  Department,  NOTS. 
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where  D;  =  a?  o2  -  (a  )2 .  None  of  the  D£  will  be  zero,  for  if  D{  =  0,  then  the  corresponding 

Ai  Aibii* 

covariance  matrix 

OAiEi  I 

would  be  singular. 

The  unknowns  are  the  corrections  ui  =  Ax,  u%  -  A y,  and  U3  =  A z  to  be  applied  to  an  initial 
estimate  xq,  yo>  zo  °f  space  position.  Therefore  k  =  3.  We  define  the  coordinates  of  the  ith  As- 
kania  station  as  X £,  T;,  Z;  where  i  =  1,  .  .  .  r. 

Then 


(i  =  1,  2 ,  r) 


yo  -  Yi-r 

[(*0  -  *i-r)2  +  (yo  -  Yi-r)2]1'2 


(i  =  r  +  1,  .  .  .  n) 


The  qji  are  defined  as  follows: 

_ Zj  ~  zq _ 

9li'(A£-x0)2+(Zi-z0)2 


(i  =  1,  2,  .  .  .  r) 
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7li  = 


-(.Yt_r  -  x0)(Y^r  -  yQ) 


K*i-r  -  *o)2  +  (yf-r  -  yo)2  +  (2£-r  -  *o)a][(^-r  ~  *0)2  +  (^£-r  “  * O^2 


(i  =  r  +  1,  .  .  ,  n) 


72  i  =  0 


(i  =  1,  2,  .  .  .  r) 


to  = 


[tf£.r-*0)2  +  (^-*0)*] 


1/2 


[(*,-,  -  *o)2  +  (T£-r  -  y0)2  +  (Z,-r  -  z0)2] 


(t  =  r  +  1,  .  .  .  ») 


73  i  = 


AX.  -  *0) 


U*  -*0)2  +  (Zi-20)2 


(i  =  1,  2,  .  .  .  r) 


731  = 


AYi-r-y0)(Z._r  _  2q) 


[Ui-r  -  *0)2  +  Wi-r  ~  y0)2  +  (Zi-r  ~  *0)«i-r  -  *0)2  +  (Z,-r  -  20)2] 


1/2  (*  =  r  +  1»  •  •  •  n) 


The  solution  shown  here  is  ordinarily  iterated  until  all  the  corrections  u£  are  negligible.  For 
further  detail  see  Ref.  5. 


APPLICATION  TO  CURVE  FITTING  OF  SPACE-POSITION  DATA 

Suppose  that  it  is  required  to  fit  polynomials  in  the  time,  t,  to  space  position,  x,  y,  z,  in  the 

form 


x  =  u\  +  u2t  +  '  •  •  +  “fj+l 

y  =  Ulx+ 2  +  “lj+3  *  +  •  •  •  +  ul}T-i2+2tl* 

z  ~  ui1+l2+ 2  +  uli+/2+  3*  +  *  *  •  +  ui1+t2+t3+3tZ3 

'I'he  quantities  required  to  apply  the  method  of  this  report  are  obtained  below. 

The  basic  data  are  x£,  y£,  zx.  Mi,  ££  (space  positions,  variance— covariance  matrices,  and  val 
ues  of  time)  for  i  =  1,  2,  .  .  .  r,  where 


i 

^lyi 

0XiZi 

Hi  = 

CT*.-yi 

ay< 

°y£z£ 

Wl 

oyi  Zi 

^2£  J 

My  definition,  n  =  3 r,  a£  =  0  for  i  =  1,  2,  .  .  .  3r, 
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Hence,  by  inspection, 


Then  the  vector  of  polynomial-equation  coefficients  is  given  by  Eq.  8,  and  the  variance- 
covariance  matrix  of  these  coefficients  is  given  by  Eq.  9.  The  smoothed  x,y,z  values  obtained  by 
evaluation  at  time  i,  together  with  the  corresponding  variance— covariance  matrix,  are  easily  ob¬ 
tained.2 

REFERENCES  TO  OTHER  LOCAL  APPLICATIONS 

The  required  quantities  for  application  of  the  method  of  this  report  to  several  specialized 
least-squares  procedures  are  easily  obtained  from  Ref.  7  and  8  and  several  informal  reports3 4'4,5. 

It  must  be  admitted  that  these  solutions  would  not  be  affected  by  the  use  of  the  method  of  this 
report.  This  is  true  because  the  assumption  of  uncorrelated  data  is  valid  in  terms  of  present 
knowledge  about  the  measurement  techniques  involved.  However,  the  possibility  does  exist  that 
the  examples  under  APPLICATIONS  can  all  be  made  special  cases  of  the  general  method.  Fur¬ 
thermore,  a  general  least-squares  subroutine  based  on  the  method  of  this  report  could  easily  be 
written  for  the  IBM  7090  computer. 

SUMMARY 

A  least-squares  procedure  for  correlated  data  has  been  presented.  The  use  of  this  method  will 
not  affect  present  solutions  unless  the  assumption  of  un correlated  data  is  to  be  replaced  by  an  es¬ 
timate  of  the  variance— covariance  matrix.  The  method  of  this  report  is  aoplicable  at  present  to  (1) 
the  use  of  derived  azimuths  and  elevations,  which  are  correlated,  to  obtain  space  position;  and  (2) 
the  fitting  of  space-position  data  by  polynomials  in  time  where  the  variance— covariance  matrix  for 
each  given  time  is  known.  It  is  recommended  that  a  general  least-squares  subroutine  incorporating 
the  equations  of  this  report  be  programmed. 


2  An  informal  report,  IDP  1339,  entitlerl  "The  Determination  of  the  Variances  and  Covariances  of  Line- 
of-Sight  Angular  Rates  as  Obtained  From  Askania  Data,’'  by  Otto  Neall  Strand,  was  issued  by  NOTS  5  De¬ 
cember  1961. 

3 Informal  report,  Technical  Note  303-26,  entitled  “COTAR  Data  Reduction  and  Error  Analysis,"  by 
Otto  Neall  Strand,  was  issued  by  NOTS  in  September  1957. 

4 1  DP  1272,  entitled  “A  Least-Squares  Star  Calibration  of  the  FLR  Camera,"  by  Otto  Neall  Strand  and 
Lee  Thomson,  was  issued  by  NOTS  on  26  June  1961. 

SlDP  1313,  entitled  “A  Least-Squares  FLR  Solution  for  Aircraft  Space  Position,”  by  Otto  Neall  Strand 
and  Lee  Thomson,  was  issued  by  NOTS  on  12  July  1961. 
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